Cyclic policy distillation: Sample-efficient sim-to-real reinforcement learning with domain randomization

Deep reinforcement learning with domain randomization learns a control policy in various simulations randomized physical and sensor model parameters to become transferable the real world zero-shot setting. However, huge number of samples are often required learn an effective when range is extensive due instability updates. To alleviate this problem, we propose sample-efficient method named cyclic distillation (CPD). CPD divides into several small sub-domains assigns local each one. Then policies learned while cyclically transitioning sub-domains. accelerates through knowledge transfer based on expected performance improvements. Finally, all distilled global for sim-to-real transfers. CPD’s effectiveness sample efficiency demonstrated four tasks (Pendulum from OpenAIGym Pusher, Swimmer, HalfCheetah Mujoco), real-robot, ball-dispersal task. We published code videos our experiments at https://github.com/yuki-kadokawa/cyclic-policy-distillation.